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Difficulties of Scientific Programming 

Computational science presents a host of challenges 
for the field of knowledge-based software design. Sci- 
entific computation models are difficult to construct. 
Models constructed by one scientist are easily mis- 
applied by other scientists to problems for which 
they are not well-suited. Finally, models constructed 
by one scientist are difficult for others to modify or 
extend to handle new types of problems. Existing 
knowledge-based scientific software design tools, such 
as SIGMA (Keller & Rimon 1992), provide only lim- 
ited means of overcoming these difficulties. For ex- 
ample, SIGMA facilitates model construction by pro- 
viding scientists with high-level data-flow language 
for expressing models in domain-specific terms. Al- 
though SIGMA represents an advance over conven- 
tional methods of scientific programming, it supports 
only certain aspects of the model development pro- 
cess. In particular, SIGMA focuses mainly on au- 
tomating the process of assembling equations and 
compiling them into an executable program. Con- 
struction of scientific models actually involves much 
more than the mechanics of building a single compu- 
tational model. In the course of developing a model, 
a scientist will often test a candidate model against 
experimental data or against a priori expectations. 
Test results often lead to revisions of the model and a 
consequent need for additional testing. During a sin- 
gle model development session, a scientist typically 
examines a whole series of alternative models, each 
using different simplifying assumptions or modeling 
techniques. A useful scientific software design tool 
must support these aspects of the model development 
process as well. In particular, it should propose and 
carry out tests of candidate models. It should analyze 
test results and identify models and parts of mod- 
els that must be changed. It should determine what 


types of changes can potentially cure a given nega- 
tive test result. It should organize candidate models, 
test data and test results into a coherent record of 
the development process. Finally, it should exploit 
the development record for two purposes: (1) auto- 
matically determining the applicability of a scientific 
model to a given problem; (2) supporting revision of 
a scientific model to handle a new type of problem. 
Existing knowledge-based software design tools must 
be extended in order to provide these facilities. 

An Artificial Intelligence Approach 

We are attacking this problem using two related ideas: 
First, we are building a “Model Development Tool- 
box”. The toolbox will support a set of generic model 
development steps that are taken by most scientists 
in the course of developing scientific computational 
models: Examples of such generic model building 

steps include: (1) mapping equations onto physical 
situations; (2) fitting models against experimental 
data; (3) testing models against experimental data; 
(4) testing applicability of models to given inputs; 
and (5) modification of models in response to test 
results. Second, we are designing a “Model Devel- 
opment Record”. The record will contain machine 
readable documentation of the entire model develop- 
ment process. To begin with, the record will describe 
the goals the model is intended to fulfill. For example, 
this might include a representation of the questions 
the model is (and is not) intended to answer. The 
record will also describe the sequence of candidate 
models that were constructed in the course of devel- 
oping the final model. For each candidate model, the 
record might describe: (1) the equations encoded in 
the model; (2) assumptions underlying the model; (3) 
fitting techniques used to instantiate free parameters 
of the model; and (4) tests against empirical data that 
were performed on the model. The record must also 
describe (5) the temporal sequence of candidate mod- 
els as well as (6) logical dependencies between test 
results on early models and modeling choices made 
in constructing subsequent, more refined models. 
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Tools for checking applicability of scientific models 
to new problems will rely heavily on the model de- 
velopment record. Important applicability checks in- 
clude: determining whether a proposed use of a model 
is consistent with the goals the model was originally 
intended to fulfill; determining if a new problem lies 
within the range of inputs for which the model was 
tested; and testing assumptions underlying the equa- 
tions that were incorporated into the model. Each 
of these checks requires access to various aspects of 
the model development record. Likewise, tools that 
support model revision will also rely heavily on the 
model development record. Important types of model 
revision include: extending/modifying the model to 
handle a wider/different range of input parameters; 
re-fitting free parameters of the model to new empir- 
ical data; changing the assumptions used to model a 
physical process; adding/deleting physical processes 
to/from the model; and changing the overall purpose 
of the model. A model revision tool should automati- 
cally determine when a revision is needed (e.g., by de- 
termining that a new problem falls outside the range 
of problems handled by the original model, or by de- 
tecting discrepancies between empirical data and out- 
puts of the model). It should suggest changes to the 
model that have the potential to cure the problem 
(e.g., by reasoning about sensitivities of outputs with 
respect to changes in intermediate results, or by rea- 
soning about the effects of potential changes in as- 
sumptions on the outputs of the model). Finally the 
system should assist in re-validating the new model, 
(e.g., by suggesting new tests of validity, and carrying 
out and evaluating such tests.) In many cases, models 
may be revised by “replaying” a portion of the devel- 
opment record that led to the original model. Replay 
will require access to logical dependencies among test 
results and modeling choices found in the develop- 
ment record, using techniques similar to derivational 
analogy (Mostow 1989) and transformational imple- 
mentation (Balzer 1985). 

System Architecture 

The overall architecture of our envisioned system is 
shown in Figure 1. The model development toolbox 
serves as a front end to the whole system. The tool- 
box interacts with a human user to build an initial 
model in some scientific domain. It also interacts with 
a user in order to revise an existing model to handle a 
new situation. Finally, the toolbox also includes facil- 
ities for controlling the application of scientific mod- 
els. As the toolbox guides the user through a series of 
model building, testing and revision steps, it interacts 
with several data bases. The model fragment data 
base contains the basic building blocks of scientific 
models. The toolbox uses techniques embodied in 



Figure 1: Model Development System Architecture 

the SIGMA system to combine model fragments into 
one or more “current working models”. As working 
models are constructed, they are tested against test 
data drawn from a test data base. Likewise, as tests 
are run, results are incorporated back into the test 
data base. As the initial model development process 
unfolds, the toolbox leaves a structured trace of the 
process in the model development record. Later on, 
the scientist will apply the model to specific problems 
in which he is interested. As the model is applied to 
each problem, the system consults the model develop- 
ment record to determine whether the model is valid 
for the current problem. If the model fails to ap- 
ply, the scientist may use the toolbox to revise the 
model. During the revision process, the toolbox is 
also guided by the model development record. The 
toolbox and record is being implemented as an exten- 
sion to the SIGMA scientific model building system 
(Keller & Rimon 1992). Testbed domains for this 
research include planetary atmosphere modeling and 
ecosystem modeling problems used in development 
of SIGMA. Additional testbed domains include two 
problems under investigation in computer-aided de- 
sign research at Rutgers University: modeling of jet 
engine nozzle performance and modeling the motion 
of sailing yachts. 

Controlled Application of Models 

Implementation of the model development toolbox 
and record is initially focusing on methods for control- 
ling the application of scientific computation mod- 
els. To this end, we have developed a collection 
of techniques that prevent users from applying sci- 
entific models to situations which violate their im- 
plicit assumptions and lead to erroneous or mean- 
ingless results. Some of these tests can be applied 
to virtually any scientific model. Such generic test 
include: (1) comparing inputs, outputs or intermedi- 
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ate results to fixed bounds; (2) verifying expectations 
about monotonicity or uni/multi-modality of com- 
puted functions; (3) validating results in comparison 
to simplified models. We have also defined a collec- 
tion of more specialized tests, whose relevance de- 
pends on the specific idealizations, approximations or 
abstractions that were used to construct the model. 
Examples include: (4) checking nearness to the fit- 
ting point of a linear approximation and (5) verifying 
self-consistency of solutions obtained by decomposing 
systems of equations, among others. 

Our applicability testing techniques require that 
models be represented in a manner that makes ex- 
plicit what tests are required and how the tests should 
be applied. For this reason, we have developed and 
implemented a model representation language that 
contains applicability checking information. Our rep- 
resentation is an extension of the dataflow graphs 
used in SIGMA (Keller & Rimon 1992). The repre- 
sentation includes annotations that describe what ap- 
plicability tests should be carried out at model execu- 
tion time. The annotations are linked to the dataflow 
graphs in a manner that allows the system to deter- 
mine the stage of the computation at which each ap- 
plicability test should be carried out. We have also 
defined and implemented a general model execution 
procedure that refers to the annotations to perform 
the required applicability tests during the course of 
model execution. We have implemented and tested 
several versions of a jet engine nozzle performance 
model and a yacht velocity prediction model in the 
new representation along with applicability tests suit- 
able to each. 

An example of a scientific model represented as a 
dataflow graph is shown in Figure 2. This graph rep- 
resents a model for computing the steady state veloc- 
ity of a sailing yacht as a function of several geometric 
and physical parameters of the yacht, (e.g., vertical 
center of gravity (VCG), wetted surface area (WSA), 
longitudinal second moment (LSM), effective draft 
(T e //)), as well as inputs describing the sailing con- 
ditions, (wind-speed (VJ«,) and heading angle (B tw )). 
The model describes a computation that proceeds in 
two stages. The first stage is to solve the torque bal- 
ance equation N etTorque{<t>) = 0 which asserts that 
“heeling” torques (causing the yacht to heel over in 
the wind) are equal to “righting” torques (causing 
the yacht to remain upright). The solution value of 
<f> is the heel angle at which the yacht will sail. The 
second stage is to solve the force balance equation 
NetForce(v , <j>) — 0 which asserts that “thrust” (due 
to the wind acting on the sails) is equal to “drag” 
(due to the friction caused by water). The solution 
value of v is the steady state velocity of the yacht. 
The “Torque Balance” and “Force Balance” nodes 
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Figure 2: Yacht Velocity Model Dataflow Graph 


of this graph each describe submodels of the overall 
yacht velocity prediction model. Each submodel is 
itself represented by a dataflow graph that describes 
a process of solving an equation using a numerical 
root-finding algorithm (Brent’s method). This yacht 
velocity model is only an approximation of a more 
accurate model of the yacht’s motion. The more ac- 
curate model solves for velocity v and heel angle <f> 
simultaneously using a pair of coupled torque bal- 
ance and force balance equations. The coupling is 
due to the fact that N etTorque actually depends on 
both <f> and v, just as NetForce depends on <f> and 
v. The coupled model is generally more accurate; 
however, it takes longer to run. It is also more brittle 
than the uncoupled model since it uses a two- variable 
equation solver ( Newton- Raphson) which more often 
fails to find a root than the two one-variable (Brent) 
equation solvers used in the uncoupled model. 

The yacht model dataflow graph illustrates our ap- 
proach to representing applicability tests as annota- 
tions to dataflow graphs. Some types of tests are 
general enough to apply to virtually any numerical 
model of a physical system. Examples of this type 
include testing whether a model exhibits a qualita- 
tive behavior that can be described in terms of mono- 
tonicity, unimodality or multimodality of the func- 
tion computed by the model. These qualitative tests 
are represented as special slots appearing in each 
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dataflow node object. For example, in the “Mono- 
tonicity” slot, an entry of the form (Input, Sign ) 
(where sign is one of {>,>,<,<} indicates that the 
model’s output is expected to be monotonic (increas- 
ing, strictly increasing, decreasing, strictly decreas- 
ing) in the named output. For example, the mono- 
tonicity slot in the “Force-Balance” object includes 
the entries (V tw , >) asserting that the velocity output 
v is expected to be a strictly increasing function of the 
wind speed V tw . Whenever a model is executed, the 
execution procedure examines the monotonicity and 
modality slots, extracts descriptors of the expected 
qualitative behavior, and tests whether the current 
execution of the model is consistent with that behav- 
ior. The current execution is checked by examining 
a database of results of previous model executions 
and verifying that the current results bear the cor- 
rect qualitative relationship to previous results. 

Some types of tests are highly specialized, and ap- 
ply only to a small number of models, perhaps only 
one model. We represent these tests as special “ap- 
plicability checking nodes” that are directly wired 
into the dataflow graph. An example of this type 
is the “Consistency Test” node in the yacht model 
dataflow graph. The consistency test checks whether 
the decoupling of the torque balance and force bal- 
ance equations is a good or bad approximation. It 
does so by evaluating the solution values of <f> and v in 
the inequality NetTorque(v , ) < K. This test mea- 
sures whether the approximate solution brings the net 
torque close enough to zero. By representing applica- 
bility tests as additional nodes in a dataflow graph, 
our system allows arbitrary computations to be used 
for applicability tests. 

Although applicability checking nodes are repre- 
sented in the same manner as the main stream of the 
computation, they are not handled in the same fash- 
ion by our model execution procedure. To begin with, 
applicability nodes are kept separate from ordinary 
nodes. Under the control of the user, the system can 
execute the entire graph, include applicability checks, 
(running in “restricted mode”) or the system can exe- 
cute only the subgraph representing the main stream 
of the computation (running in “unrestricted mode” ). 
Furthermore, our execution procedure allows the ap- 
plicability checking nodes to determine whether or 
not execution should be aborted in the event of an 
applicabilty failure. Outputs of applicability tests are 
typically routed to a special “Enable” input of other 
nodes. When an applicability test disables another 
node in the graph, all computations downstream of 
the test are aborted. 

Initial tests of our system for controlling applica- 
tion of models have demonstrated two types of ben- 
efits. When provided with inputs that would previ- 


ously have caused models to return erroneous results, 
our system returns an error condition indicating that 
the model is not applicable to the current input. The 
system thus avoids misleading the user with erroneous 
results. In addition, our system informs the user of 
which applicability tests failed and thus makes him 
aware of the reason the model does not apply to the 
current input. In the future, we plan develop tools for 
using such diagnostic information to support revision 
of scientific models to change or extend their ranges 
of applicability. 

Summary 

The model development toolbox and record is in- 
tended to support a variety of activities that occur in 
the course of developing scientific computation mod- 
els. These activities include construction and test- 
ing of new models; controlled application of models 
to specific problems, and revision of models to han- 
dle new situations. The system is also expected to 
promote rapid development of new scientific compu- 
tational models, more reliable use of scientific models 
among computational scientists; wider sharing of sci- 
entific models within communities of scientists; and 
deeper understanding among scientists of the assump- 
tions and modeling techniques incorporated in the 
models they use. A more detailed description of this 
research is found in (Ellman 1993). 
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